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Abstract —Ultra-wide area FTV is an FTV with very wide viewing zone where motion parallax is 
realized. 4D orthogonal ray-space is analyzed and applied to ultra-wide area FTV. Ray-space of “a 
group of rays through one point” is derived in 4D orthogonal ray-space. It is extended to obtain 
ray-space captured by linear arrangement cameras. View generation of ultra-wide area FTV needs 
rays that are not captured by real cameras. These rays are synthesized by interpolating the 
captured ray-space so that the intersections of the captured ray-space and the ray-space of rays 
emitted from a light source have the same color. 


1 INTRODUCTION 


FTV (Free-viewpoint Television) [1]-[8] is visual media that transmits all ray information of 
a 3D space and enables immersive 3D viewing as if we were actually there. Omnidirectional 
FTV is an FTV with very wide field of view (FOV), that is 360-degree FOV. Ultra-wide area 
FTV is an FTV with very wide viewing zone that gives motion parallax. FTV was developed 
based on ray-space representation [9]-[12]. 4D ray-space is needed for immersive 3D viewing 
with full parallax. 

We proposed two types of ray-space, orthogonal ray-space and spherical ray-space. 
Orthogonal ray-space is used for linear and planar camera arrangements and spherical ray-space 
is used for circular and spherical camera arrangements. 4D spherical ray-space was analyzed and 
applied to omnidirectional FTV [13]. In this paper, 4D orthogonal ray-space is analyzed and 
applied to ultra-wide area FTV. Ray capture and view generation in 4D orthogonal ray-space are 
presented. 


2 4D ORTHOGONAL RAY-SPACE 


2.1 4D Orthogonal Ray-Space of Rays through One Point 


Fig. 1 shows the definition of orthogonal ray-space. As shown in Fig. 1, one ray is expressed 
by intersection (x, y) and direction (0, g) on a reference plane. Therefore, one ray is expressed by 
one point in 4D parameter space (x, y, 0, g). This parameter space is 4D orthogonal ray-space. 
Each point in the 4D ray-space has an intensity f of the ray. Therefore, fis expressed as f(x, y, 0, 
p). 

Here, “a group of rays through one point” is considered as shown in Fig. 2. This concept is 
used for ray capture, view generation and ray synthesis as shown in Fig. 3. Fig. 3(a) shows ray 
capture by a real camera, and view generation by a virtual camera collecting rays through one 
point. Fig. 3(b) shows ray synthesis. If the surface of an object is a diffusion surface, one point 
on the surface emits rays with equal magnitude. Therefore, if one ray is known, other rays can be 
synthesized. 
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Fig. 2 Concept of “a group of rays through one point”. 
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Fig. 3 Ray capture, view generation and ray synthesis by using the concept of “a group of rays 
through one point”. 


x and y of a ray with direction (0, g) through a point P(X, Y, Z) are given by 


x=X — tanOZ (1) 
-yy lang 
yes cos a 2) 


(1) and (2) give the 4D orthogonal ray-space (x, y, 0, y) of a group of rays through one point. By 
changing 0 and ọ, this group of rays forms a plane in (x, y, 0, g) space. To visualize it, (x, y, 0) 
space with fixed g and (x, y, g) space with fixed 0 are considered in the following. 
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From (1) and (2), the following equations are derived. 


Ts 5 6) 
1+ (tang)? = (ae) (4) 


For fixed g, (1), (3) and (4) give the top view, front view and side view of (x, y, 0) space, 
respectively. The top view is a straight line with slope -Z and both the front and side views are 
hyperbolic curves in (x, y, tan0) space. Therefore, a group of rays through one point forms a 
hyperbolic curve in (x, y, tan0) space with fixed g as shown in Fig. 4. Three examples of 


hyperbolic curves for o > 0, g=0 and ø <0 are shown. 
For fixed 6, (1) and (2) form a straight line in (x, y, tang) space as shown in Fig. 5. Three 


examples of straight lines for 0> 0, 6=0 and 0< 0 are shown. 


Z>0 
@ fixed 


Fig. 5 Rays through a point P(X, Y, Z) form a straight line in (x, y, tang) space with fixed 0. 
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2.2 4D Orthogonal Ray-Space Captured by Linear Arrangement Cameras 


Let arrange many cameras on a line that is set on the xz-plane (y=0) and parallel to the x-axis. 
When an omnidirectional camera at P(X., 0, Z.) on the line captures a ray with direction (0, g) as 
shown in Fig. 6, x and y are given by 


x= X, — tanOZ, (5) 


y= = tang=Z.tang J1 + tan’0 (6) 


(5) and (6) give ray-space captured by linear arrangement cameras. 
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Fig. 6 Ray capture by linear arrangement cameras 


By varying X., (5) and (6) form a hyperbolic cylinder in (x, y, tan@) space with fixed ọ as 
shown in Fig. 7. They form a plane in (x, y, tang) space with fixed @ as shown in Fig. 8. The 
expressions of the hyperbolic cylinder and the plane are given by 


y=Z.tang, /1 + tan’ foranyx (7) 


and 


Ze 


cos 


y= tang for any x (8) 


respectively. 
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Fig. 7 Ray-space in (x, y, tan@) space with fixed ọ captured by linear arrangement cameras. 
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Fig. 8 Ray-space in (x, y, tang) space with fixed 0 captured by linear arrangement cameras. 


3 VIEW SYNTHESIS FOR ULTRA-WIDE AREA FTV 


4D orthogonal ray-space is applied to ultra-wide area FTV [11]. Ray capture and synthesis 
for ultra-wide area FTV is shown in Fig. 7. Rays are captured by many cameras on a line (y=0, 
z=Z,) and virtual camera views are synthesized from the captured rays. Let one of the rays 
captured by a virtual camera be (xo, vo, 9, go). This ray is synthesized because it is not captured 
by real cameras. When z of the light source of this ray is assumed to be Z, the position (X, Y, Z) 
of the light source is determined. This light source emits rays in various directions. Let ray-space 
of the emitted rays be (x, y, 0, 9). 
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Fig. 9 Ray capture and synthesis for ultra-wide area FTV. 


Some of emitted rays are captured by the real cameras. These rays are given by the 
intersection of emitted and captured ray-spaces. Fig 10 shows captured ray-space, emitted ray- 
space and their intersection in (x, y, g) space with fixed 0. The intersection is given by 


(x, y, tang) = (X - Ztand, 5, z (7) 


Many intersections are obtained by varying 0. Fig. 11 shows the trajectory of this intersection. 
: f; LY á $ * 
It is on a horizontal plane of y= —. When Z is correct depth, the intersections have the same 


color. This color is given to the ray (xo, yo, 90, Yo). 


Captured raytspace 


Fig. 10 Captured ray-space, emitted ray-space and their intersection in (x, y, g) space with fixed 
0. 
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Fig. 11 Trajectory of the intersection of captured ray-space and emitted ray-space in (x, y, 9) 
space by varying 0. 


The process of this ray synthesis is summarized in the flowchart of Fig. 12. 
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Fig. 12 Flowchart of ray synthesis. 
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4 CONCLUSION 


Ultra-wide area FTV with full parallax is realized by 4D orthogonal ray-space processing. 
Ray-space of “a group of rays through one point” is derived in 4D orthogonal ray-space. It is 
used for ray capture, view generation and ray synthesis. Ray-space of “a group of rays through 
one point” forms a hyperbolic curve in (x, y, tan) space with fixed g and a straight line in (x, y, 
tang) space with fixed 0. By varying the camera position along a line, ray-space captured by 
linear arrangement cameras is obtained. This ray-space forms a hyperbolic cylinder in (x, y, tan) 
space with fixed g and a plane in (x, y, tang) space with fixed 0. 

Rays for view generation of ultra-wide area FTV are synthesized by interpolating the 
captured ray-space using the concept of “a group of rays through one point”. It is done so that the 
intersections of the captured ray-space and the ray-space of rays emitted from a light source have 
the same color. 
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